8D-THERMO CAM: Combination of Geometry with Physiological Information for Face Recognition

Biometrics-based technologies in the area of identity management are gaining increasing importance, as a means of establishing non-falsifiable credentials for end users. However, in the three-way tug-of-war between convenient, unobtrusive data collection (required for user acceptance), accuracy in results (required for justifying deployment), and speed (required for widespread use in practice), no single biometric to date has managed to hold the middle ground that would allow for its ready adoption. The overall goal of our project is to develop the theoretical framework and computational tools that will lead to the development of a practical, unobtrusive, and accurate face recognition system for convenient and effective access control. This framework will encompass 8D characteristics of the face (3D geometry + 2D visible texture + 2D infrared texture, over time). In this video, we present a novel multi-modal facial recognition approach that employs data from both visible spectrum and thermal infrared sensors (Fig 1). Data from multiple cameras are used to construct a 3D mesh representing the upper body and a thermal texture map. We have constructed a subdivision surface using anthropometric statistics, which serves as a parametric model of the human face. This model is aligned and fitted to the data using the elastically adaptive deformable model–based fitting framework [1]. From the fitted parametric model we extract two images corresponding to the subject’s face: • a three channel parametric deformation image encoding geometry (by recording the displacement of the corresponding face model point) and • a one channel parametric thermal image encoding temperature. We subsequently process these images to extract biometric signatures. Specifically, the deformation image is compressed using a wavelet transform and the vasculature graph is extracted from the parametric thermal image. Recognition is accomplished by comparing the signatures obtained from: 1) the parametric deformation image, 2) the parametric thermal image, and 3) the visible spectrum texture map. The novelty of our work lies in the use of deformation images and physiological information as means for comparison [2]. By combining deformation images and vasculature graphs as metadata, our algorithms can overcome changes in pose or appearance from the time of enrollment, including facial expressions. We have performed extensive tests using the Face Recognition Grand Challenge datasets [3] and our own multimodal database with very encouraging results. The latest updates can be found at the following URL: www.vcl.uh.edu/UR8D